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growing ccccert with accctjctahility of 
sental b'ealth service ageLOie^^ increased eifhasis is bein^ focqseg 
cc prcgraia ^ eif-^aluation«. EvaluaticD^ criteria invclve systeis^ 
-sanaqeaent, client utiliza^icB, and cctccse cf intervention. In 
designing training ror self -evaluayticn , cn^ lust deaJ. with issues 
raised by the nefc audience of potential trainees arc the new content 
'ai^eas in wbicn skills' are ^ n eeded.^ Five gaps in ccrrent evaluation 
setfacSs .incl ude the need to provide a 'high vcluse cf trainicg at 
little cost to particjipants; the need fci individualized, 
agency-specific assistance; ^ the anility tc trair perscn^s firos greatly 
diverse roles and background; the need for training in t'he social 
skills fcr aanaging the huisan side cf evaltfatic4; and the nee-d to 
integrate tne social in ter aqjional ccipcnent of evaluaticn with the 
technical tocls'that are readily avaiiatle. The Evaluaticn Curriculus 
Develcpaent Project developed ^ aodel cf trainicg in self -evalu ation 
to Beet these nee^s* It contains five lesic units: (1) assessing - 
brc/'dniz aticnal readiness for evaluaticn; <2) reaching consensus on 
^he definitions of progras evaluation; (3) identifying organizational 
prerequisites tc evaluation ; develcpirg an- overview cf the » 

evaluation cycl^; and (5) pinpointing evaluative guesticns*^ to be . 
answered* ' (Author/Bii) 
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Training to Facilitate Agency Self-Evalua^tion 

. ^ ^ \ ^ • / . 

Amid the growing concern with .accountability of mental health service 

agencies, increased e^hasis is being focused on one type o< tooUfor achiev- 
ing accountability progfam self-evaluation. *" * , 

Self-evaluation, or iRternal evaluation, is defined .here as evaluation 
£haft is plaphed^ carri^/out/ and used by persons inside the agency or 
system being evaluated^ While the Impetus to undertake the evaTuati^ may 
come from outside the system, persons within the agency must be invoUed in 
evaluation planning in order to cdhduct self-evalyation successful ly-. Simi- 
larly, while outside audiences such as state or federal sponsors may also 
use the information; a substantial portion c/f^ the data collected' must have 
utility for internal priogram p-lanning. 

The growing emphasis on agency self -evaluation in mental health is 
illustrated by the provisions of the Community total Health Center ^end- 
men|;s-of 1975 (P. I. 94-63). Here, self-evaluation is one of three genertTf* 
tyfJes of evaluation required of federally funded CMHCs, and guidelines furthe 
specify three evaluative levels within the self-evaluation category: systems 
manag^nt, client utilization, and outcome of intervention.^ Consider the 
variety of evaluation activities this Involves. 

" At the systW management level, P.L. 94-63 requires centers to (1) evfl 
uate the cost of operations by units of major types of direct and indirect 
services, (2) review th? "numbers and rates of catchment area residents' use 
^f /the^centef's services, by element of service and client age, sex, family 
income, race dnd geographic subarea, (3) define the availability of services, 
and (4) assess theinental health needs of persons within the catchnient area, 
/it the client utilization leve?, centers are asked to evaluate patterns of 
use of services,. including (1) arwareness of services ^)y local residents, (2) 
acceptability of.semces, defined as a predeliction by residents to use and 
c^ntifiue to use, ^ center, and (3) accessibility of services, defined in 
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terms of temporal ^ geographic, financial; psychological and sociocultural 
accessibility. The third 1 eve! ^. outcome of intervention, designate? evalua- 
tion of the impact .of .center services on the mental health of the residents 
of the catchment area* This category includes evaluation- of the effective- 
ness and quality of both direct and indirect services; e.g., studying the 
impact of outpatfent services on the relief of clie/its presentirvg problenis^ 
documen^g effective deduction in appropriate institutional izatipr^ tfjro^h' 
referral and screening,, aftercare and community placement, X 

. Vihile some of these' Evaluation activities are pres^d^f being carried out 
by the CMHCs (activities largely o'n the systems' mac>^g^nt eategory, accord- 
idg to available data) mar\y of these self-evaj^cfati^n fufictions represent 
♦ new undertakings. Often, new skills arey^^qiflred 'by persons in the field to' 
carry out the additional' evaluatior^^l^d n>tes and functions. The need 
for new skills fias, in; .turn ^ epsSted* an .enormous market for training in 
self-evaluation/ 

In designing tra^Hiing for self-evaluation, Gtie must deal with issues 

^audience of potential trainees and*{2) the new content, 
areas in whiyr^sktlls are needed. First, the- natui^ of the^training audience 
^places demands or); tbe content and format of Sel'f-evaluatipn trailing.. Poten- 
tial trainees in sjelT-evaluation are estabi ished professionals in the mental 
hpalth' field, often with a wide range of expertise, albeit expettise that' 
#loes not necessarily include evaluation. Nevertheless th6 mental health pro- 

0 

fessional already possesses a range of clinical and adjiiinistrattVe skills 

that cafj be brought to bear oh evaluation as well as a knowledge of the system 

iri which the evaluation will be carried 5ut. The critical importance of'the 
* * , ' • 

latter will be taken up again later in this paper. Another important cojlsider^^ 
tion relatejl to audience characterlstic^is the fact that evaluation is l>eing ^ 



' raised by (1 ) the. 
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added onto thef alrea^ay fuU schedul^ of persons in the field, frequently 
with HtHe or no Increase .in' resources* These persons' concern with evalua«» 
tion is not academic; it stems from a rfeed for problem-solving to satisfy ^1 
the 'demancte of tbeir job situations. In training, these- s)ersons und^stand- 
ably have little patience for learning aboat "evaluation" in an exhorta- 
tory sense; instead, tiiey press for, practical "how-to-do-it" in^ructions ^ 
that ^hey can ^pply to -their own settings. Providing concrete 'instructions 
for designing, performing and using specifit evaluations is often complicated 
by the variety of roles represented in the audience. As Wilier points out, 

the audiences of self-evaluation may fill a number of roles within the agency 

• ^ » * 

3 

from director to line staff or consumer. Jloreover, in the federally funded 
Centers, ^these roles may be carried out in the c^te;>k of any of eleven ser- 
vice subsystems : inpatient care, outpatient care, emergency services, 
services to the elderly, .services to children, transitional serviys, day 
care, consultation and education, screening and referral, drug abuse re.lated ' 
services and alcohol abuse related services. In sum, the audience for train- 
ing in self-pvarTljations is typically composed of seasoned professionals from 

« 

a variety of service components and organizational roles, needing concise, 
practical instruction on how to plan, carry out and use specific evaluation 
techniques in their agencies. Meeting t^e training needs of these persons 
offers a particular challenge. 

The second major 'consideration i^ designing^self-evaluation training 
is the nature of the new content areas in which skills are needed* The 
task -at hand fs no longer evaluation done by persons oatside a mental health 
agency, but rather' building an evaluation system within , e,g> , a CHHC* This 
shift in. focus broadens the required skills considerably beyond thdse of 

► c 
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research expertise tp'>nclude skills for handling the particular task of 
placing what have been r^ditional ly seen as "research" activities in a con- 

. text that has been tradi tiaijaTly devoted to service delivery , not research! 
To prepare for such a, task, on^ musi^ for exampl e , ' draw upon principles of 
plaoned change and org'^nizational neadiness ,for innovation, the innovation 
in this case being evaluation. An ajiequate design for self-eval;i.ation in- 

.eluded not only research strategies, but strategies for ^overcoming organi- 
zational resistance and bui Iding support for the evaluation process. Parti- 
cular mana^eriaV skills such as team-building, participative decision-making, 
structuring coitmunication may be critical to the development of ^elf-evalua- 
tion. In s'lim, the .content areas it^ which persoYis -require training for 
self-evaluati.on include both technical^esearch skills and social-interaction- 
al skills. ^ * ... . - 

Keeping in mind these Characteristics of the trainigg^ audfence and the 

/needed content areas, what means are currently available to develop e^er- 
tise in evaluation? The Evaluation Curriculum Development Project at the 
University of Michigan collected close to va hundred examples of evaluation 
training programs^ b^roadly defined. While the information* was ^largely self- 
reported by the programs themselves ar^d the sample is by ho means inclusive, 
it does suggest some general descriptors of the format and content of current 

trainfng. The format of training generally falls into one of five categories: 

. ■ . • 

(a) formal degree programs, (b) exten'ded institutes, (c) workshops and con- 
ferdnces, (d) consultation to .ifidiyldual agencies, and te) technical assistance 
through jnanuals, publications, etc. Tjbe number of formal degree programs- 
available .in evaluation has multiplied rapiqly in recent years, lihile the 
majority of these *a re masters. ^nd doqtoral programs, specialized trairting in 
evatQdtion is also available at the bachelors and postdoctoral levels. 



I P)[*ograms generally 'range in length from 2 ttf 4 years and are centered on 

university campuses. ClearTy this format offers the opportunity for the . 
, . Extensive. training in evaluation; however, it does little to increase the 
^ expertise of persons already in the^field. The ^econd format, extended in« . 
stitutes, attempts to provide intensive 'graining in a reduced time frame. 
Professionals are brought to a universl'ty campys for 5-8 weeks of special-, 
l| ized training, then retwmed to'theit' agencies. At least one of the summer 

institutes was designed to include some followup of persoRs' subsequent work 
in the field. Persons attending these programs, are generally at the post- 
nasters or post-doctorate level. Since registration in these programs is 
\ highly ^elective> it is also /lot adequate to meet the scope of, training 

^needed, The third format, workshops* or conferences is $V far the most conmon 
method of offering training to* professionals already in the field. Workshops 
are generally sponsored by universities or professional groups. and are held 
^ on "neutral" ground such ^s a hotel or conventioo center. Length of work- 
shops typically ranges from 1 to 3 days. While these programs are reasonably 
accessible to field persons, they su-ffgr the limitations of presenting a 
great deal c^F information in a very brief time. 'This task is made even more 
difficuJt^by the^ heterogeneity of the audiences.> In all of the above formats 
a given training audience is likely composed of persons from a variety of* 
agencies and ^professional disciplines. The fourth format, consultation, 
- moves the trainers into a particular agency context, addressing the pro^J^s 
or information needs of persons |(n^that setting. * The spec ifi^ ^^oxg^frfzat ion 
of the consul t5ttion may vary from a workshop including many or aH staff to 
% Individual discussion with one 9r two staff. Consultations on evaluation are 

typically cocopleted in- one' or two days, depending on, the comple>;ity ^f th? 
^ .problem being' addressed and the resources Available to the agency. While' , 
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consultation offer? the most individual ized* instruction's it is often not avail- 
• able, to agencies with lijnited resources. The final category, technical assistance 
^through manuals and publications, while not "training" in a strict sense neverthe- 
less merits attention as one the najor avenues open to persons who need to-. . 

/ , ■ \' ^ , • 

develop their evaluation expertise. UViH has given particular attention to the ' 

• • • 

y/ritten TOdium iji providing technical .as?li stance .to CMHCs implementing P.L 

hj 4 • ^ ^ 

94-63. Literature on progrfn evaluation has multiplied^ rapidly in re£ent 

years, including numerous publicXtions that attempt to provide prescriptive 

steps'for evaluators. Of .cours^ wi-itten training materials can only be useful , 

if they are read, and here utilization presents some real problems. Most 

*• written materials assure a base level of knowledge that suggests to the reader 

why and wheo he/she might want to read a particular 'document. Lac);ing basic 

grourrdwork on evaluation, many persons find it difficult to'begin via written 

" ^ 'niaterials. . • . 

- * ' ^ ^, • * • 

The content of current training in evaluatf4on is characterized by an em- - 

' *\ phasis on technical ^research skills and relative inattention to procedures f or 

integrating evaluation .activities with the ongoing operations of a service 

delivery ^eircy: "^ie individuals may become well-acquainted with specific 

evaluation tools, the, problem of matching .tools to particular agency neecjs and/or 

^iritegratin^^^iscrete tools^with one. another to build a self-evaluatiag system 

often goes unaddressed by. training.^ Morepver, the human side of evaluation — 

jiandlipg resistances, gaining support from^dministration'^nd cooperation from^ 

line staff, getting people to work together to plan and carry ou^ the deita 

collection and then actually' use the information 7- is largely ignored, in favor 

of bolsterinjg people's research skilT^. This content focus is cqnsistent with^ 

the longstandtng assumption that evaluati(in is basically a research activity.' 

' ' j. ■ . ■ ^ ^ 

As evaluation is increas-in'gly viewed as 'a management aquvtty, ev^luatjon train- 
.lag content must undergo a similar shift in .focus. / ' / 

ERIC • . • . -8 •.: *■ • . • . : 



'. In sum. when one examines the characterfstfcs of current evaluation train- 
• . • • . ■ ' ' 

ing methods in light of the audiences requiring triaining and the content 

areas'in whfch informatit)n is -needed, five specific gaps emerge. First, 'there 
is a^ need to provide a high volume of training at littfe cost to participants. 
Workshops offer one alternative <pf meetihg this needjb/here a seconcT gai^ 
emerges: the need for individualized, agency-specific a,ssistance that can be. 
applied directly to a gi|?en mental health setting. Third, persons from great-' 
^ ly diverse roles and background will pe participating in the self-ev^uation 
process but little traiciing^as attempted to. prepare t jjj^ sometimes "strange 
bedfeTlov)s: for working together. Fourth, training in social sHins for man- " 
aging the human side of evaluation has been almost totally neglected in defer- 
ence to technical research skills/ Fifth, there is a need to give S^e^y 
persons a self-evaluation framework that integrates the social interactional 
component of evjaluation with the technical tools that are already available. • 

The Evaluaia.Dn^Curriculum Development Project set out'to deJelop a model . 
of training in self-evaluation to meet these identified tra'Th'ing needs. At 
its 'present stage of development, the model addresses the problems of bringing 
agency persons to a common starting point and providing' a framework for. working 
through the, initial steps of evaluation planning as a group. The. model accom- 
plishes these tasks through ,ftye basic training units: . , ' 

I. Assessing organizational readiness for evaTuation 

II. Reaching consensus pn the definitions of program ^ ' ^ 

' evalaation / * 

IIL^ Identifying organizational prerequisites to evaluation 
Iv. Present an overview of the evaluatioff cycle 
V. Pinpointing fevaluative qaestiorft to be answered ^3 
These trairt^ng units provide a forvtn for learning four major types of 
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grqupMctivities involved in self-evaluation: {)) reaching a commoitiaroup 
un^standing of infom^titfn, (2) reaching groap consensus, (3) prioritizing 
needs ^ a group, and (4) group planning for action. - : 

'The ftrst unit of the tT^aining model assessing orgafiizatlonal readiness 
foe evaluation ,, is ^cot a training activity per se but an information retrieval 
activity. The underlying assumption here is that the developmertt of self- 
evaluation requires certain descriptive •information c^onceming the 4ys tern being 
eva-luated. One must understand where the program is starting fPom before be- 
ginning to1)uild evaluation capacity. A nimiber of fnodels for assessing org- 
anizational readiness for evaluation are available to mental health agencies; 
however, one of the most informative assessments relevant to internal evalua- 
tion is that originated by Howard/Oavis at NIHH. Davis identified eight dimen- 
sions influencing readiness for evaliiation, popularly summarize^ by the acronym 
AVICTORY: Ability, Values, Idea, CircumstaTices, Timing, Obligation, Resistance 
Yield. The PrdjrT*&m» Evaluation 'Resource Center (P.E.R.G.) directed by Thomas 
Kiresuk has developed a quantitative instrument to assess categories' of readi- 
ness withirr each .of "the AVICTORY ams. For- a minimefT'fee, PfRC will provide 
agency staff with*, copies of the instruraent, .score the completed questionnaires 
and give the agency a profile of its readiness Vor evaluation. These data (or 

^ * . comparable inforlfeition obtained by an alternate means)* lay the groundwork for 

* * * 

^ ' 5 • 

educating ^taff <ii^ the process or self-evaluation. ^ • - , 

• The- second, unit of the model, reaching consensus 'on^'the definitions of 
* • 

program evaluation , brings agency staff togeffjer in the first working session-. 
Optimal group size should" -not exceed ten persons; therefore multiple groups 
would t)e required in l^rge agencies. Sub-group composition might sample per- 
sons vertically or horizontally in the organizational structdre. The purpose 

m * 

of this training unit is to focus agency, staff ipetnbers on tbe question, "What 

t . 
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*do^s-evali/atibn mean?" Structured gsoup discussion will allow all members to 
view and hear responses to this general, question* All^group members will have 
the opportunity to express their. thoughts- on the sutyject. . Subse^iuerttly, ' 
Staff members, through^ structured group consensus exercise, will develop 
^' v«>rking definition of evaluation which is both acceptable and understandable 
to all^mbers. There are a variety of definitions and purposes that can be . 
assigned to the term, "evaluation." The unit is not intended to adybcat6 
any specific definition, but rather to encourage*all members of the group to 
4ir their perceptions, biases, and opir>ions on the topic. A major intention of 
this unit^is to open lines of communication between agency staff at all organi- 
zational levels and to establfsh the legitimate involvement of aU group members 
•in the self-evaluation process. TJie intragroup exchange among individuals and 
the airing of thmights on the subject of evaluation is an essential part of 
team-building. The definitions of all agency subgroups would be summan^zed 
and fed back to the total agency. _ Th^logic of this second unit is. twofold: 
(1) the literature and our own evaluation .training experience suggest that 
sta.ff memb.ers have 'differing' conceptions of the def-initions and purposes of 
evaluation, an^that this difference can be a source of confusion and conflict. 
Clearing the, air about the nature a^d purpose of self-evaluation is thus the 
first logical step in the group training process; and (2) tiie unit allows 
the participants to practice two group facilitators: ^reaching.a common under- 
standing of .information" and reaching group consensus. 

The third unit, identffylriq organizational prerequisites tb>evaluation , 
first involves analyzing the resul*ts of the readiness assessment carried out 
in 'unit one. After' viewing the data «nd sharing interpretations of the mean- 
ing of the. information,. participants are dividpd into smalj groups' accordiqg 
to their job functions in the agency j(e.g,, "administration," *and "counselling") 
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> • ^ • f ^ 

Each small/group performs the fallowing .activities: ^ 

*{1) identify the two strongest characteristics, according" to- the readiness 
. profile, which will fatili-tate^ the agency's undertaking of. evaluation 

at 'that point in- time; . . , ' 

•• V (2) identify the two weakest Chafacteri sties wh-ich will work against 
the agency's undertaking 'evaluation at th'fe point 'in time; and , 

■ .'" • -« • » ■ ' '• 

(3), brjftn|tonn suggested sOlUtiians for overcoming these 'weak chara'cteris- 

\tics. ' . ■ ' • • . ^ 

Participants would^then reconvene in theMai^ge group in order to develop group 

^plans for 'changing these characteristics. The development of group plans / 

would be facilitated^^, a structi^igd pjanning format. At the end of this* 

unit,*^t*ie part;ic7pants will have ^developed their own plan for strengthening 

their ageijcy's ability to undertal?e ev.a|luation as well as having part'icipated 

in the group processes or priojMtizing needs and group p>ahnint|^r action, 
■* _ • ' \ 

J Only ^after these preliminary steps have been taken to prepare agency per-* 

sol|iel for working through self-evaluation together, should the overall evalua- 

tion process, including research procedures, be introduced, fraditiona^^evalua 

tion training, you will recall, begins witf>.this unitt)ut, in ignoring the pre- 

paratory activities, too often gives' agency persons iafoi;in^ti^n that they are^ 

unable to' use/ The proppsed trainihg^^del avoids this pitfall by giving 

initial attention^^to Organizatiorfal c6nte)ct\ Unit four, overv.iew of thyself- 

evaluation cycl'fe , ther^ moves from a^ focus on prjjfess to a f6cus*on contentT^^ 

First, participants are introduced to the 't:Dtal cj(cle of self-evaluation.- Any 

4 

of a number of stepw'ise evaluation procedures described in the literature 
* .• ' 

could be used here. Taking as an example the evaluation process' Jaid Out by 

' ^. -fi ' ' 

the A.D. Little manual, the steps might be: • * ' ■ ' 

♦Choose topics to ft/aluate • - * ' 

\ ^ ' * ' ' 

♦Define the boundaries of topics to evaluate 



*l3esign the evaluation - • ^ ^ :. 

"^"^onduct the evaluation* . / ^ ' . * v 

^Analyze the ^evaluation data ' , - . 

♦ / 

♦Present .the results df the eval union - .' " 
r,^ *Enhance utilization o'f resujts ^. . * ? 



■J- 

The second activity of unit four is to revievr the group facilitator*activtr 
/^ties pat participants have practiced in units two and £Kree^ Finally,' the^ 
training materials for unit foiur will^discuss the link between the overall- self- 
evaluation process and the group facilitator activities. The purpose here is 
to have participants experience ithe linkage between social and technical as^ 
.^pects of self-evalua.tion. % 

The final uhft* of the training model developed to date addresses the * 
task of pinpointing evaluative questions , yith this unit;' participants J^egin 
to fiK)v6 through the technical steps of the generic eval^tion cycle. The 
parpose'fcf this/unit ife to'help agency personnel iden^if^ the Information 
ttiey need in orders to improve thei'r programs. The^ot£j^yjit5i.ve of this module .is 
to facititate-the use of self-evaluation results inT)Tanning ^gjd de^sion mak- 
ing fay collecting information that is relevant to persons at multfple pro- ' ^, 
gram levels. 'vThis module will" provide participants with tools to gather tnpuj 
from the relevant subsystems, cff the organylzation^ (e.g. , prp^ram 'units) con- 
^ ceming (.1) operational goals, (2^ general information needed to engage in 
planning and decision making vis a vis 1iJ^|e goals, (3) types of questions to 
ask to gather the needed information, (4) potentia^J information sourdes, and 
' (5) how the information*^ should be communicated to the.subunit in order to be 
useful to those persons (preliminary Input). A structured group* process will 
be designed to facilitate communication between individuals and group manage- 
ment of information^ . # . - 
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Having identified the evaluation' questions that need- to be answered 

will put the agency in a position to-link with extant evaluation resource . 
♦ • * 

.materials that focus on technical research tools. 

^The proposed model of training to facilitate agency self«evaluation spe- 
cifically a.ddresses the five' gaps in' current training previously identified. 
First, the model offers potential for use as. a low-budget, higff volume train-- ' 
ing vehicle. Since the mpdeTs stepwise^'format and its accompanying '.instruc- 
tional units are self-contained, it could be used within an agency in a va^r- 
iety of ways. If resources* are not a concern, an individual consultant could 
'be^^^'ught in -to -guide agency personnel through each unit* The model is 
easily adaptable to a low-budget dissemination approach, however. Using a 
'"snowball" approach, agency representatives <ould be introduced to -the model 
^a centralized workshop, then returned to their agencies where they, in 
turn, wbuld guide a staff group in working through each unit. In large agen- 
cies, additional groups could be led byjnembers of the original group. Since 
the' written materials would be made available to all staff,, the'role of the 
"leader" in each group is minimal; he/she acts morels a group facilitator 
thaH as an instructor. Potentially, this model can provide low-budget train- 
ing while still addressing the secoad*gap^ the rieed f^or ageocy-specific assis- 
tance. . ' / 

The proposed model builds on the^evel of readinelss for evaluation of a 
specific agency, by collecting this information before undertaking any tra^ining. 
Jhe training occofs within the agency, as with the*more costly consultation- 
model, and is directly applicable to the eval^tion concerns df that setting,'^ 
Persons are given a structure for beginning the actual self-evaluatior^planning 
procedure in tbe course of training. Third, this model is specificall^j^esigned 
to coriyene the diverse audiences that are involved in self-evaluation --a. Board 



njemb^rs,' director, middle mr\ageme\t persons, line staffs and clijant consumers 
— and provide a group process for bringing .thesa persons to a^ common star^ting 
glace and helping thefji to work through the initial planning stages.^together. 
Fourth, the modeT^as presently dev^oped, provides both instruction and exper- 
• ience in four social skills 'that sq^rve to facilitate the self-evaluation process. 
(1) reaching a common group understanding of informatioli, (2) reaching group ^ 
consensus, (3) prioritizing needs as a group, and (4) group planning for action 
Fifth, the model provid'es for the integration of the social and'. technical 
aspects of evaluation, building upon the wealth of technical tools already 
available. 

In closing, I would like "to touch bcJefly on some of the problems involved 
in evaluating an ^evaluation training program such as'^the proposed model. In a ^ 
recent review of methodologies for evatuating training programs, the Evaluation 
Curriculum Development Project found that whiT^e most training was designed to ^ 
change participants' knQv/ledge or behavior, the majority of programs were 
evaluated on the basis^f participant self-reported satisfaction with tra'ining. 
These findings are less than sui;^rising in light of the difficulty of designing 
behavioral measures to evaluate training. In the model presented hei^ fo^ 
example, one is assuming that the proposed materials and group training format 
will result in the participants' ability to perform certain evaluation tasks y 
and that the performance of these tasks will- in turn result in^ agency self- 
evaluation capacity. To evaluate this model, one should design behavioral . 
measures to test th6 linkage between training and task performance and between 
task performance and self -evaluation capacity. Moreover, in order to demon- 
strate causality, the agencies trained uoder this rltodel sljould be compared 
with control agencies who availed themselves of otiier types of training. (Given 
th6 current pressure to evaluate, virtually all mental health agencies' are 

r 

seeking some means of increasr^ng their expertise so there is qperati^nany no- 
sUch thing as a "no-treatment" control group*) . . , . • 



•. ■ * . - ■ \. ^- . * 

The. linkage, between training artd task performance can be exarnined by 

build^ behavioral tasks into the training itself. In the present^model .for 

example, participants demoastrate their mastery of certain group*proCess 

skillsiin the course of completing ttte assignment of a given unit. The link- 

age between skill acquisition and self^^aluation capacity. is more •difficult 

to demonstrate. Even a miaimaf des<;r/t>tion^ of changes in self-e\taluation capac- 

ity is mre difficult to dempnstrate. Even *a m<irH?nal description of changed 

in self-evaluation would require longitudinal study of a number of ^organiza- 



tional 'Variable%. Here one begins .to run into questions of the cost/benefit 
ratio of conducting impact evaluations ^f train-rng over tame. Currently in 
evaluatiro^ training, we are trying'to learn more about the shape of 'developing 
self-evaTuatton capacity so that we can^ make more intelligent judgments of 
what changes we woiJld logical ly^ expect to occur^in a given time period follow- 
•ing training. Attention should be given to irpproving the quality of the 
evaluation of training hand Tn han^l with striving to i^nprove training content . 
Mhert the impact of training on participant behavior is not assessed one 
should.be aware, of the limitations .of the cwiclusioos that can be drawn. 
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